1. Field of the Invention
The present invention relates to an image processing apparatus and method and a computer program. More specifically, the present invention relates to an image processing apparatus and method for correcting for color variations in a mosaic image (multi-view image) produced by joining a plurality of images photographed from different viewpoints, and to a computer program.
2. Description of the Related Art
Panoramic pictures or star maps are produced by joining camera pictures photographed from different viewpoints. An object may be photographed from different viewpoints using a single camera or a plurality of cameras.
An object may be photographed using a single camera that is manually or automatically pointed in many directions to obtain multi-view images of the object, and the images may then be joined. In this case, the object is not photographed from a plurality of viewpoints at the same time. Thus, basically, an object is photographed using a single camera under the condition that the object is stationary.
However, when the photographic environment, e.g., the lighting conditions, changes while photographing a stationary object, changes in hue occur in the photographed images. In a picture, such as a panoramic picture, produced by joining the images including such changes in hue, the boundaries of the joined images become noticeable.
In another method, an object is photographed using a plurality of cameras that are pointed in different directions from a plurality of viewpoints at the same time, and the plurality of photographed images are then joined. In this method, a moving object can be photographed, which is not possible by using a single camera; however, a problem of color variations due to the difference among the cameras still exists.
Due to the difference in characteristics of, for example, a lens, a data capturing unit, and a memory, and the difference in image processing parameters of the cameras, the same object photographed using the cameras is not stored as the object having the same color data. In a composite picture in which these images are merged, therefore, the difference in hue among the images becomes noticeable.
At boundaries of the joined images, unnatural discontinuous color fields appear, and the image boundaries are exaggerated. Thus, a high-quality mosaic image, such as a panoramic picture, cannot be obtained. In particular, a composite picture made using a plurality of cameras having different characteristics is more susceptible to such color variations, which are not negligible compared to the change in hue caused as the single-camera environment changes over time.
In order to overcome color variations in a mosaic image produced by joining a plurality of photographed pictures, one known approach is to superpose and average the overlapping regions. This approach will be described with reference to FIGS. 1A to 1C.
FIGS. 1A and 1B show pictures of an object photographed from different viewpoints, and FIG. 1C shows a panoramic picture produced based on the pictures shown in FIGS. 1A and 1B. The object is photographed so that the pictures shown in FIGS. 1A and 1B include an identical object scene region, called an “overlapping region”.
The pictures shown in FIGS. 1A and 1B include overlapping regions 11 and 12 indicating an identical scene region.
When the panoramic picture shown in FIG. 1C is produced as a mosaic image based on the pictures shown in FIGS. 1A and 1B, color correction is performed so that an overlapping region 21 in the panoramic picture shown in FIG. 1C has the mean color value of the overlapping regions 11 and 12 of the pictures shown in FIGS. 1A and 1B. This color correction results in a panoramic picture in which boundaries of the adjacent pictures are not noticeable.
Color correction may be performed using a technique other than such a simple pixel-averaging correction. Correction using a weighting and averaging procedure may be performed, taking distances 31 and 32 from the pictures into account, so that the amount of pixel value correction can vary depending upon the distances 31 and 32.
A variety of image processing procedures based on “overlapping regions” are disclosed in the art. For example, Japanese Unexamined Patent Application Publication No. 5-342344 discloses that color conversion is performed using an overlapping region so that color histograms of two color images for the overlapping region are matched, to thereby match the hue of the two images.
Japanese Unexamined Patent Application Publication No. 2000-92306 discloses a method for compositing two CCD images including an overlapping portion at the image boundaries, in which the amount of offset in a mean value of pixel data histograms of the two images for the overlapping portion is added to the images, thereby reducing color variations in the boundaries of the joined images.
Japanese Unexamined Patent Application Publication No. 2001-169151 discloses that a plurality of images photographed using a single camera so as to include an overlapping region are joined to form a composite image that is less noticeable at the boundaries of the joined images by adjusting the hue, brightness, etc., at the boundaries.
In the “overlapping region”-based correction disclosed in the above-noted publications, adjacent images must include an identical scene region or an “overlapping region.” Without an “overlapping region,” correction cannot be performed. A larger overlapping region is required for correcting for worse color variations between adjacent images. If the “overlapping region” is small, it is difficult to accurately perform correction.